feat: Add custom domains and split-face architecture to the bootstrap server by hansl · Pull Request #764 · dfinity/sdk

hansl · 2020-06-29T21:31:23Z

This introduces a series of changes related to bootstrap and how it operates;

Split the HTTP Agent into a generic interface and a few files in their own folder,
In general, stop referring to httpAgent in favor of agent, and use the interface
Create a Proxy Agent and a ProxyStub class to abstract marshaling Agent API calls,
Remove any non-marshallable objects from the Agent API (basically just Response objects which cannot be transformed into JSON) in favor of just the properties we need,
Bootstrap build now outputs 6 files; index.html/index.js (the main bootstrap code), worker.html/worker.js (the IFRAME worker code that keeps the localStorage and talk to the IC) and candid.html/candid.js (Candid UI).
Added instructions on how to develop with Bootstrap.
Added information about the startup process of Bootstrap. These might need some updates after Tungsten
Rework bootstrap build dependencies (upgrade version etc)
Move most of the bootstrap utility functions related to the bootstrapping process to a shared class that detects location and whether a worker is needed or not. Also createAgent function now can return an IFRAME worker or an actual agent, depending on context.
webpack.config.js builds 3 entrypoints that are all independent of each others (before bootstrap was expected to be the only entry point).

Some changes are lateral but general QoL for 0.5.10:

window.icHttpAgent has been deprecated for a while and replaced with window.ic.agent. More properties will be added to window.ic. This includes exposing a GlobalInternetComputer interface that can be applied to Window/Global objects for typing.
upgrade the version and configuration of prettier for JavaScript.
Some reformatting due to the prettier upgrade.

andrewwylde

Most of my notes are questions for my lack of knowledge, but this looks good to me

andrewwylde · 2020-06-29T22:30:54Z

src/bootstrap/src/index.ts

-  }
-  return defaultValue;
-}
+declare const window: GlobalInternetComputer & Window;


This and this should be the same, no? Probably ideal for its own file or something.

It's not. If you declare const window: GlobalInternetComputer, window doesn't have any other properties (like window.location). I could extend GlobalInternetComputer with Window, but then it doesn't fit with global (global doesn't have location).

andrewwylde · 2020-06-29T22:33:37Z

src/agent/javascript/src/actor.ts

 export interface ActorConfig {
  canisterId?: string | CanisterId;
-  httpAgent?: HttpAgent;
+  agent?: Agent;


Is there a reason we're moving to "agent" instead of httpAgent? I don't totally follow

httpAgent is the Agent implementation that talks over HTTP. In this PR we introduced ProxyAgent that talks to a proxy function, and we could have more agents (MockAgent, SmithAgent, etc)

Ahh okay, nice

andrewwylde · 2020-06-29T22:45:47Z

src/agent/javascript/src/actor.ts

-        return agent.retrieveAsset(cid, path);
+        const arg = IDL.encode([IDL.Text], [path]) as BinaryBlob;
+        return agent.query(canisterId, { methodName: 'retrieve', arg }).then(response => {
+          switch (response.status) {


do we want to have default case do something similar to this, or do we control the agent.query response type? Perhaps adding a little logging so that if you get an unexpected response type, we can log it. I figure that'd avoid issues like you fixed the other day :)

The object is built from the public spec, so there is only two status possible (see https://github.com/dfinity-lab/ic-ref/blob/master/spec/index.adoc#read-query-call)

andrewwylde · 2020-06-29T22:48:52Z

src/agent/javascript/src/agent/http.ts

+
+    // Run both in parallel. The fetch is quite expensive, so we have plenty of time to
+    // calculate the requestId locally.
+    const [response, requestId] = await Promise.all([


[Naivity ahead]: Since we're returning metadata, should error handling happen here, or should it be handled by where submit() is invoked?

The Metadata is useless unless the submit succeeds, so we should break if there is an issue. Since we build our own messages, it's unlikely that hashing the request ID would fail (it's more likely that the fetch will fail, for example).

andrewwylde · 2020-06-29T22:50:55Z

src/agent/javascript/src/agent/proxy.ts

+      case ProxyMessageKind.RequestStatusResponse:
+        return resolve(msg.response);
+      default:
+        throw new Error(`Invalid message being sent to ProxyAgent: ${JSON.stringify(msg)}`);


andrewwylde · 2020-06-29T22:55:29Z

src/agent/javascript/src/agent/proxy.ts

+  }
+
+  public createCanister(principal?: Principal): Promise<SubmitResponse> {
+    throw 2;


Is this to ensure it's implemented? Not sure if abstract class is a useful paradigm here, just figured I'd ask to learn more

Actually I should fix those before we release Tungsten. I'll do some follow up stuff.

andrewwylde · 2020-06-29T22:59:42Z

src/bootstrap/README.adoc

+your own nginx reverse proxy. Look up instructions online.
+
+**Note that HTTPS is needed for the crypto API if you're accessing a non-localhost URL. This is
+a limitation of the web API (see )**.


I think you meant to link to an anchor here?

andrewwylde · 2020-06-29T23:10:16Z

src/bootstrap/tsconfig.json

  "compilerOptions": {
-    "incremental": true,
-    "target": "es2017",
+    "outDir": "ts-out/",


Is there a reason we stop using incremental?

I was using Webpack. Put backl.

## Changelog for advisory-db: Branch: master Commits: [rustsec/advisory-db@c5ebb272...7c81bd4f](rustsec/advisory-db@c5ebb27...7c81bd4) * [`8b068177`](rustsec/advisory-db@8b06817) Add use-after-free advisory for `yottadb 1.1.0` and earlier * [`be43bc8c`](rustsec/advisory-db@be43bc8) Assigned RUSTSEC-2021-0022 to yottadb * [`221ec336`](rustsec/advisory-db@221ec33) rand_core: incorrect check on buffer length when seeding RNGs ([RustSec/advisory-db⁠#764](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/764)) * [`7c81bd4f`](rustsec/advisory-db@7c81bd4) Assigned RUSTSEC-2021-0023 to rand_core ([RustSec/advisory-db⁠#771](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/771))

## Changelog for advisory-db: Branch: master Commits: [rustsec/advisory-db@0283f676...f960a503](rustsec/advisory-db@0283f67...f960a50) * [`2b296940`](rustsec/advisory-db@2b29694) Add advisory on nb-connect SocketAddr casting * [`d0689e37`](rustsec/advisory-db@d0689e3) Fixed sized-chunks * [`18b2916a`](rustsec/advisory-db@18b2916) Assigned RUSTSEC-2021-0021 to nb-connect * [`8b068177`](rustsec/advisory-db@8b06817) Add use-after-free advisory for `yottadb 1.1.0` and earlier * [`be43bc8c`](rustsec/advisory-db@be43bc8) Assigned RUSTSEC-2021-0022 to yottadb * [`221ec336`](rustsec/advisory-db@221ec33) rand_core: incorrect check on buffer length when seeding RNGs ([RustSec/advisory-db⁠#764](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/764)) * [`7c81bd4f`](rustsec/advisory-db@7c81bd4) Assigned RUSTSEC-2021-0023 to rand_core ([RustSec/advisory-db⁠#771](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/771)) * [`c2e8d57f`](rustsec/advisory-db@c2e8d57) Add unmaintained crate advisory for `safe-api` * [`55026f94`](rustsec/advisory-db@55026f9) Add unmaintained crate advisory for `jsonrpc-quic` * [`96b16887`](rustsec/advisory-db@96b1688) Assigned RUSTSEC-2021-0024 to safe-api, RUSTSEC-2021-0025 to jsonrpc-quic * [`3aada4c4`](rustsec/advisory-db@3aada4c) Add advisory on comrak XSS * [`a1aa2136`](rustsec/advisory-db@a1aa213) Assigned RUSTSEC-2021-0026 to comrak * [`5775f258`](rustsec/advisory-db@5775f25) Fixed memory corruption. ([RustSec/advisory-db⁠#776](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/776)) * [`b724f12a`](rustsec/advisory-db@b724f12) Update CVE numbers ([RustSec/advisory-db⁠#777](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/777)) * [`12033122`](rustsec/advisory-db@1203312) heapless: use-after-free when cloning partially consumed Iterator * [`9283d50a`](rustsec/advisory-db@9283d50) heapless: fix year: 2020, not 2010 * [`7c260c52`](rustsec/advisory-db@7c260c5) Assigned RUSTSEC-2020-0145 to heapless * [`eda0a223`](rustsec/advisory-db@eda0a22) Add an advisory on lifetime extension in generic-array * [`c71170c3`](rustsec/advisory-db@c71170c) Assigned RUSTSEC-2020-0146 to generic-array * [`fe25ca02`](rustsec/advisory-db@fe25ca0) Add advisory for out-of-bounds write in bam * [`fae90251`](rustsec/advisory-db@fae9025) Assigned RUSTSEC-2021-0027 to bam * [`af1b1de4`](rustsec/advisory-db@af1b1de) Add advisory for memory safety issue in toodee's insert_row * [`f4e465c6`](rustsec/advisory-db@f4e465c) Assigned RUSTSEC-2021-0028 to toodee * [`b4afb832`](rustsec/advisory-db@b4afb83) Report uninitialized memory exposure in truetype * [`dafd2019`](rustsec/advisory-db@dafd201) Assigned RUSTSEC-2021-0029 to truetype * [`1c7888e4`](rustsec/advisory-db@1c7888e) Update RUSTSEC-2020-0146.md with list of patched versions ([RustSec/advisory-db⁠#789](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/789)) * [`196ad433`](rustsec/advisory-db@196ad43) Heapless soundness fix since 0.6.1 ([RustSec/advisory-db⁠#791](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/791)) * [`14e6efec`](rustsec/advisory-db@14e6efe) Mark RUSTSEC-2020-0146 as unsound ([RustSec/advisory-db⁠#788](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/788)) * [`927b303d`](rustsec/advisory-db@927b303) Revert "Mark RUSTSEC-2020-0146 as unsound ([RustSec/advisory-db⁠#788](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/788))" * [`e59320ad`](rustsec/advisory-db@e59320a) Add advisory for double-free in scratchpad * [`8887e3da`](rustsec/advisory-db@8887e3d) Assigned RUSTSEC-2021-0030 to scratchpad * [`d9614925`](rustsec/advisory-db@d961492) Add advisory for uninitialized memory drop in byte_struct * [`50db2cc7`](rustsec/advisory-db@50db2cc) Add advisory for aliasing violation in nano_arena * [`6dc26382`](rustsec/advisory-db@6dc2638) Assigned RUSTSEC-2021-0031 to nano_arena * [`b8b93bd6`](rustsec/advisory-db@b8b93bd) Assigned RUSTSEC-2021-0032 to byte_struct * [`a0c9d676`](rustsec/advisory-db@a0c9d67) Add advisory for double-free in stack_dst * [`2e968911`](rustsec/advisory-db@2e96891) Assigned RUSTSEC-2021-0033 to stack_dst * [`aced392f`](rustsec/advisory-db@aced392) Add unmaintained advisory for rulinalg crate * [`25e792c0`](rustsec/advisory-db@25e792c) Add unmaintained advisory for office crate. Fixes [RustSec/advisory-db⁠#742](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/742) * [`240080a1`](rustsec/advisory-db@240080a) Assigned RUSTSEC-2021-0034 to office * [`a0ec5819`](rustsec/advisory-db@a0ec581) Add advisory for invalid layout assumptions in quinn ([RustSec/advisory-db⁠#804](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/804)) * [`e93938cb`](rustsec/advisory-db@e93938c) Assigned RUSTSEC-2021-0035 to quinn ([RustSec/advisory-db⁠#805](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/805)) * [`0f0dbd06`](rustsec/advisory-db@0f0dbd0) Assigned RUSTSEC-2020-0147 to rulinalg * [`ac5a6f92`](rustsec/advisory-db@ac5a6f9) Report 0162-internment to RustSec * [`423ca504`](rustsec/advisory-db@423ca50) Assigned RUSTSEC-2021-0036 to internment * [`4f30ae8e`](rustsec/advisory-db@4f30ae8) Report use-after-free issue in diesels sqlite backend * [`e5c10bcc`](rustsec/advisory-db@e5c10bc) Update crates/diesel/RUSTSEC-0000-0000.md * [`c31f016d`](rustsec/advisory-db@c31f016) Update crates/diesel/RUSTSEC-0000-0000.md * [`dd560c65`](rustsec/advisory-db@dd560c6) Assigned RUSTSEC-2021-0037 to diesel * [`ae1107e4`](rustsec/advisory-db@ae1107e) Update rustsec-admin version to use new website generator ([RustSec/advisory-db⁠#810](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/810)) * [`a33c05d2`](rustsec/advisory-db@a33c05d) add fltk advisory * [`5ce40487`](rustsec/advisory-db@5ce4048) Update RUSTSEC-0000-0000.md * [`d07de997`](rustsec/advisory-db@d07de99) Update RUSTSEC-0000-0000.md * [`35e0ea59`](rustsec/advisory-db@35e0ea5) replace with keywords * [`a84dbc5d`](rustsec/advisory-db@a84dbc5) formatting * [`805270dc`](rustsec/advisory-db@805270d) Assigned RUSTSEC-2021-0038 to fltk * [`3205a091`](rustsec/advisory-db@3205a09) Report 0090-endian_trait to RustSec * [`ef52d55d`](rustsec/advisory-db@ef52d55) Report 0109-arenavec to RustSec * [`c2248b0e`](rustsec/advisory-db@c2248b0) arenavec: update advisory title to clarify issue * [`f2f59e4e`](rustsec/advisory-db@f2f59e4) Assigned RUSTSEC-2021-0039 to endian_trait * [`0bc081bb`](rustsec/advisory-db@0bc081b) Assigned RUSTSEC-2021-0040 to arenavec * [`0487b3fc`](rustsec/advisory-db@0487b3f) README.md: fix "Report Vulnerability" button ([RustSec/advisory-db⁠#818](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/818)) * [`ad84df90`](rustsec/advisory-db@ad84df9) Mirror 'main' branch to 'master' ([RustSec/advisory-db⁠#819](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/819)) * [`2c43b700`](rustsec/advisory-db@2c43b70) Rename `master` branch to `main` ([RustSec/advisory-db⁠#820](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/820)) * [`7b10b2e7`](rustsec/advisory-db@7b10b2e) Fix `main` -> `master` mirroring ([RustSec/advisory-db⁠#821](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/821)) * [`f960a503`](rustsec/advisory-db@f960a50) Have master-to-main mirror force push ([RustSec/advisory-db⁠#822](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/822))

wip

a937ace

hansl changed the title ~~wip~~ feat: Add custom domains and split-face architecture to the bootstrap server Jun 29, 2020

hansl added 2 commits June 29, 2020 14:37

prettier

78db56d

prettier

cd76cea

andrewwylde approved these changes Jun 29, 2020

View reviewed changes

comments

30ec8bf

hansl marked this pull request as ready for review June 29, 2020 23:32

hansl requested a review from a team as a code owner June 29, 2020 23:32

Merge branch 'master' into hansl/bootstrap-tungsten

56aed55

hansl added the automerge-squash label Jun 30, 2020

mergify bot merged commit bafd92f into master Jun 30, 2020

mergify bot deleted the hansl/bootstrap-tungsten branch June 30, 2020 00:43

mergify bot removed the automerge-squash label Jun 30, 2020

Conversation

hansl commented Jun 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewwylde left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hansl commented Jun 29, 2020 •

edited

Loading